Efficient Estimation of Regularization Parameters via Downsampling and the Singular Value Expansion Downsampling Regularization Parameter Estimation
نویسندگان
چکیده
The solution of the linear system of equations Ax ≈ b arising from the discretization of an ill-posed integral equation with a square integrable kernel H(s, t) is considered. The Tikhonov regularized solution x is found as the minimizer of J(x) = {‖Ax − b‖2 + λ‖Lx‖2} and depends on regularization parameter λ which trades off the fidelity of the solution data fit and its smoothing norm, determined by the choice of L. Here we consider the case of L = I, and employ the moment method to provide the relationship between the singular value expansion and the singular value decomposition for square R. Renaut School of Mathematical and Statistical Sciences, Arizona State University, P.O. Box 871804, Tempe, AZ 85287-1804, Email: [email protected] M. Horst Department of Mathematics, The Ohio State University, 100 Math Tower, 231 West 18th Avenue, Columbus, OH 432101174, E-mail: [email protected] Y. Wang Department of Mathematics, The Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong, E-mail: [email protected] D. Cochran School of Electrical, Computer and Energy Engineering, Ira A. Fulton Schools of Engineering, Arizona State University, P.O. Box 875706, Tempe, AZ 85287-5706 E-mail: [email protected] J. Hansen School of Mathematical and Statistical Sciences, Arizona State University, P.O. Box 871804, Tempe, AZ 85287-1804, E-mail: [email protected] integrable kernels. The resulting approximation of the integral equation permits examination of the properties of the regularized solution x(λ) as a function of the resolution of the system. Thus we prove that estimation of the regularization parameter can be obtained from a down sampled representation of the system of equations. Resolution and statistical arguments provide the relation for the regularization parameters across scales. Hence the estimate of λ for a large problem may be found by downsampling to a smaller problem, or to a set of smaller problems, effectively moving the costly estimate of the regularization parameter to the small scale. Moreover, the complete singular value decomposition for the fine scale system is replaced by a number of dominant terms which is determined from the coarse resolution system, again reducing the computational cost. Numerical results illustrate the theory and demonstrate the practicality of the approach.
منابع مشابه
Large-scale Inversion of Magnetic Data Using Golub-Kahan Bidiagonalization with Truncated Generalized Cross Validation for Regularization Parameter Estimation
In this paper a fast method for large-scale sparse inversion of magnetic data is considered. The L1-norm stabilizer is used to generate models with sharp and distinct interfaces. To deal with the non-linearity introduced by the L1-norm, a model-space iteratively reweighted least squares algorithm is used. The original model matrix is factorized using the Golub-Kahan bidiagonalization that proje...
متن کاملEfficient Estimation of Regularization Parameters via Downsampling and the Singular Value Expansion
The solution, x, of the linear system of equations Ax ≈ b arising from the discretization of an ill-posed integral equation g(s) = ∫ H(s, t)f(t) dt with a square integrable kernel H(s, t) is considered. The Tikhonov regularized solution x(λ) approximating the Galerkin coefficients of f(t) is found as the minimizer of J(x) = {‖Ax− b‖2 +λ‖Lx‖2}, where b is given by the Galerkin coefficients of g(...
متن کاملStochastic Downsampling for Cost-Adjustable Inference and Improved Regularization in Convolutional Networks
It is desirable to train convolutional networks (CNNs) to run more efficiently during inference. In many cases however, the computational budget that the system has for inference cannot be known beforehand during training, or the inference budget is dependent on the changing real-time resource availability. Thus, it is inadequate to train just inference-efficient CNNs, whose inference costs are...
متن کاملA Newton Root-Finding Algorithm For Estimating the Regularization Parameter For Solving Ill-Conditioned Least Squares Problems
We discuss the solution of numerically ill-posed overdetermined systems of equations using Tikhonov a-priori-based regularization. When the noise distribution on the measured data is available to appropriately weight the fidelity term, and the regularization is assumed to be weighted by inverse covariance information on the model parameters, the underlying cost functional becomes a random varia...
متن کاملAutomatic estimation of regularization parameter by active constraint balancing method for 3D inversion of gravity data
Gravity data inversion is one of the important steps in the interpretation of practical gravity data. The inversion result can be obtained by minimization of the Tikhonov objective function. The determination of an optimal regularization parameter is highly important in the gravity data inversion. In this work, an attempt was made to use the active constrain balancing (ACB) method to select the...
متن کامل